本文档提供了SNACS的详细语言描述(Adposition和Case Supersenses的语义网络; Schneider等,2018),这是52个语义标签(“ Supersenses”)的库存,这些标签(“ Supersenses”)表征了在某种程度上使用ADIP定位和案例标记的使用。粒度水平,如Streusle语料库中所示(https://github.com/nert-nlp/streusle/;版本4.5 track track track offelines guidelines guidelines版本2.6)。尽管SNACS的库存渴望成为普遍的,但该文档是特定于英语的。其他语言的文档将单独发布。版本2是Schneider等人对英语提出的超音库存的修订。 (2015,2016)(此后为“ V1”),这又基于以前的计划。本清单是在对英语的V1语料库注释进行广泛审查后开发的,以及以前未分析的属格案例所有人(Blodgett和Schneider,2018年),并考虑了希伯来语,印地语,韩国和德国的定义和案例现象的考虑。 Hwang等。 (2017)介绍了V2方案的理论基础。 Schneider等。 (2018)总结了该方案,其应用于英语语料库数据以及自动歧义任务。刘等。 (2021)提供了一个英语词法语义识别标签仪,其中包括SNACS标签的输出。该文档也可以与Xposition网站上的语料库数据一起浏览(Gessler等,2022):http://www.xposition.org/
translated by 谷歌翻译
Code generation models have achieved impressive performance. However, they tend to be brittle as slight edits to a prompt could lead to very different generations; these robustness properties, critical for user experience when deployed in real-life applications, are not well understood. Most existing works on robustness in text or code tasks have focused on classification, while robustness in generation tasks is an uncharted area and to date there is no comprehensive benchmark for robustness in code generation. In this paper, we propose ReCode, a comprehensive robustness evaluation benchmark for code generation models. We customize over 30 transformations specifically for code on docstrings, function and variable names, code syntax, and code format. They are carefully designed to be natural in real-life coding practice, preserve the original semantic meaning, and thus provide multifaceted assessments of a model's robustness performance. With human annotators, we verified that over 90% of the perturbed prompts do not alter the semantic meaning of the original prompt. In addition, we define robustness metrics for code generation models considering the worst-case behavior under each type of perturbation, taking advantage of the fact that executing the generated code can serve as objective evaluation. We demonstrate ReCode on SOTA models using HumanEval, MBPP, as well as function completion tasks derived from them. Interesting observations include: better robustness for CodeGen over InCoder and GPT-J; models are most sensitive to syntax perturbations; more challenging robustness evaluation on MBPP over HumanEval.
translated by 谷歌翻译
While pre-trained language models (LM) for code have achieved great success in code completion, they generate code conditioned only on the contents within the file, i.e., in-file context, but ignore the rich semantics in other files within the same project, i.e., cross-file context, a critical source of information that is especially useful in modern modular software development. Such overlooking constrains code language models' capacity in code completion, leading to unexpected behaviors such as generating hallucinated class member functions or function calls with unexpected arguments. In this work, we develop a cross-file context finder tool, CCFINDER, that effectively locates and retrieves the most relevant cross-file context. We propose CoCoMIC, a framework that incorporates cross-file context to learn the in-file and cross-file context jointly on top of pretrained code LMs. CoCoMIC successfully improves the existing code LM with a 19.30% relative increase in exact match and a 15.41% relative increase in identifier matching for code completion when the cross-file context is provided.
translated by 谷歌翻译
Inferring reward functions from human behavior is at the center of value alignment - aligning AI objectives with what we, humans, actually want. But doing so relies on models of how humans behave given their objectives. After decades of research in cognitive science, neuroscience, and behavioral economics, obtaining accurate human models remains an open research topic. This begs the question: how accurate do these models need to be in order for the reward inference to be accurate? On the one hand, if small errors in the model can lead to catastrophic error in inference, the entire framework of reward learning seems ill-fated, as we will never have perfect models of human behavior. On the other hand, if as our models improve, we can have a guarantee that reward accuracy also improves, this would show the benefit of more work on the modeling side. We study this question both theoretically and empirically. We do show that it is unfortunately possible to construct small adversarial biases in behavior that lead to arbitrarily large errors in the inferred reward. However, and arguably more importantly, we are also able to identify reasonable assumptions under which the reward inference error can be bounded linearly in the error in the human model. Finally, we verify our theoretical insights in discrete and continuous control tasks with simulated and human data.
translated by 谷歌翻译
In unstructured environments, robots run the risk of unexpected collisions. How well they react to these events is determined by how transparent they are to collisions. Transparency is affected by structural properties as well as sensing and control architectures. In this paper, we propose the collision reflex metric as a way to formally quantify transparency. It is defined as the total impulse transferred in collision, which determines the collision mitigation capabilities of a closed-loop robotic system taking into account structure, sensing, and control. We analyze the effect of motor scaling, stiffness, and configuration on the collision reflex of a system using an analytical model. Physical experiments using the move-until-touch behavior are conducted to compare the collision reflex of direct-drive and quasi-direct-drive actuators and robotic hands (Schunk WSG-50 and Dexterous DDHand.) For transparent systems, we see a counter-intuitive trend: the impulse may be lower at higher pre-impact velocities.
translated by 谷歌翻译
资源说明框架(RDF)和属性图(PG)是表示,存储和查询图数据的两个最常用的数据模型。我们提出了表达推理图存储(ERGS) - 构建在Janusgraph(属性图存储)顶部的图存储,该图还允许存储和查询RDF数据集。首先,我们描述了如何将RDF数据转换为属性图表示,然后描述将SPARQL查询转换为一系列Gremlin遍历的查询翻译模块。因此,开发的转换器和翻译器可以允许任何Apache TinkerPop符合图形数据库存储和查询RDF数据集。我们证明了使用JanusGraph作为基本属性图存储的建议方法的有效性,并将其性能与标准RDF系统进行比较。
translated by 谷歌翻译
大型基于变压器的预训练的语言模型在各种知识密集的任务上取得了令人印象深刻的表现,并可以在其参数中捕获事实知识。我们认为,考虑到不断增长的知识和资源需求,在模型参数中存储大量知识是亚最佳选择。我们认为,更有效的替代方法是向模型提供对上下文相关的结构化知识的明确访问,并训练它以使用该知识。我们提出了LM核 - 实现这一目标的一般框架 - 允许从外部知识源对语言模型培训的\ textit {解耦},并允许后者更新而不会影响已经训练的模型。实验结果表明,LM核心获得外部知识,在知识探索任务上的最先进的知识增强语言模型中实现了重要而强大的优于性能。可以有效处理知识更新;并在两个下游任务上表现良好。我们还提出了一个彻底的错误分析,突出了LM核的成功和失败。
translated by 谷歌翻译
感知哈希映射图像具有相同语义内容与相同的$ n $ bit Hash值相同的图像,同时将语义不同的图像映射到不同的哈希。这些算法在网络安全方面具有重要的应用,例如版权侵权检测,内容指纹和监视。苹果的神经哈什(Neuralhash)就是这样一种系统,旨在检测用户设备上非法内容的存在,而不会损害消费者的隐私。我们提出了令人惊讶的发现,即神经锤差是线性的,这激发了新型黑盒攻击的发展,该攻击可以(i)逃避对“非法”图像的检测,(ii)产生近乎收集,以及(iii)有关哈希德的泄漏信息。图像,全部无访问模型参数。这些脆弱性对神经哈什的安全目标构成了严重威胁;为了解决这些问题,我们建议使用经典加密标准提出一个简单的修复程序。
translated by 谷歌翻译
最近出现了变异推断,成为大规模贝叶斯推理中古典马尔特·卡洛(MCMC)的流行替代品。变异推断的核心思想是贸易统计准确性以达到计算效率。它旨在近似后部,以降低计算成本,但可能损害其统计准确性。在这项工作中,我们通过推论模型选择中的案例研究研究了这种统计和计算权衡。侧重于具有对角和低级精度矩阵的高斯推论模型(又名变异近似族),我们在两个方面启动了对权衡的理论研究,贝叶斯后期推断误差和频繁的不确定性不确定定量误差。从贝叶斯后推理的角度来看,我们表征了相对于精确后部的变异后部的误差。我们证明,鉴于固定的计算预算,较低的推论模型会产生具有较高统计近似误差的变异后期,但计算误差较低。它减少了随机优化的方差,进而加速收敛。从频繁的不确定性定量角度来看,我们将变异后部的精度矩阵视为不确定性估计值。我们发现,相对于真实的渐近精度,变异近似遭受了来自数据的采样不确定性的附加统计误差。此外,随着计算预算的增加,这种统计误差成为主要因素。结果,对于小型数据集,推论模型不必全等级即可达到最佳估计误差。我们最终证明了在经验研究之间的这些统计和计算权衡推论,从而证实了理论发现。
translated by 谷歌翻译
对RAS和RAF蛋白的行为与细胞膜中局部脂质环境之间关系之间的关系的了解对了解癌症形成的基础机制至关重要。在这项工作中,我们采用深度学习(DL)来学习这种关系,通过预测基于脂质膜的RAS和RAS-RAF蛋白复合物的蛋白质定位状态,该状态基于蛋白质结构域周围的脂质密度(CG),相对于脂质膜。分子动力学(MD)模拟。我们的DL模型可以预测六个蛋白质状态,总体准确性超过80%。这项工作的发现为蛋白质如何调节脂质环境提供了新的见解,这反过来又可以帮助设计新型疗法以调节与癌症发展相关的机制中的这种相互作用。
translated by 谷歌翻译